Please welcome new Cambrian-AI Analyst Gary Fritz, who contributed to this article. Artificial Intelligence applications are starting to show up in everything from cell phones to supertankers. But at the edge, they are running into the same roadblocks that traditional applications have fought for years: they need more speed. What's a burgeoning neural net to do? To make matters worse, machine learning models are growing at an exponential rate, doubling in size every 3.5 months.

In episode 88 of the IoT For All Podcast, Perceive Founder and CEO Steve Teig joins us to talk about how Perceive is bringing the next wave of intelligence to IoT through machine learning at the edge. Steve shares how Perceive developed Ergo, their chip announced back in March, and how these new machine learning capabilities will transform consumer IoT. Steve Teig is an award-winning technologist, entrepreneur, and inventor on 388 US patents. He's been the CTO of three EDA software companies, two biotech companies, and a semiconductor company – of these, two went public during his tenure, two were acquired, and one is a Fortune 500 company. As the CEO and Founder of Perceive, Steve is leading a team building solutions and transformative machine learning technology for consumer edge devices.

As AI becomes more prevalent in industry, more engineers and scientists – not just data scientists – will work on AI projects. They now have access to existing deep learning models and accessible research from the community, which allows a significant advantage than starting from scratch. While AI models were once majority image-based, most are also incorporating more sensor data, including time-series data, text and radar. Engineers and scientists will greatly influence the success of a project because of their inherent knowledge of the data, which is an advantage over data scientists not as familiar with the domain area. With tools such as automated labeling, they can use their domain knowledge to rapidly curate large, high-quality datasets.

Today NVIDIA introduced Jetson Xavier NX, "the world's smallest, most powerful AI supercomputer for robotic and embedded computing devices at the edge." With a compact form factor smaller than the size of a credit card, the energy-efficient Jetson Xavier NX module delivers server-class performance up to 21 TOPS for running modern AI workloads, and consumes as little as 10 watts of power. Jetson Xavier NX opens the door for embedded edge computing devices that demand increased performance but are constrained by size, weight, power budgets or cost. These include small commercial robots, drones, intelligent high-resolution sensors for factory logistics and production lines, optical inspection, network video recorders, portable medical devices and other industrial IoT systems. AI has become the enabling technology for modern robotics and embedded devices that will transform industries," said Deepu Talla, vice president and general manager of Edge Computing at NVIDIA. "Many of these devices, based on small form factors and lower power, were constrained from adding more AI features.

Lattice Semiconductor announced key performance and design flow enhancements for its Lattice sensAI solutions stack. The Lattice sensAI stack provides a comprehensive hardware and software solution for implementing low power (1mW-1W), always-on artificial intelligence (AI) functionality in smart devices operating at the edge. IHS forecasts 40 billion devices will be operating at the network edge by 2025. For reasons including latency, network bandwidth limitations, and data privacy, OEMs designing always-on edge devices want to minimize sending data to the cloud for analytics. Lattice sensAI enables such OEMs to seamlessly update their existing designs with low power AI inferencing optimized for their application requirements.

For decades, those four words--not to be confused with a hit Daft Punk song--have both driven fear into developers and driven sales. However, as the energy burdens for the Internet of Things (IoT), cloud computing, crypto currencies and artificial intelligence (AI) increase, a fifth word is necessary: greener. Xnor.ai (Xnor) isn't scared of the "greener" challenges facing industries today, and the unveiling of its new application-specific integrated circuit (ASIC) technologies proves so. "Power will become the biggest bottleneck to scaling AI," said Ali Farhadi, co-founder of Xnor. "What Xnor has proved today is that it is now possible to run AI inference at such low power that you don't even need a battery. This will change not only the way products are built in the future, but how entire cities and countries deploy AI solutions at scale."

Chip and system designers are giving greater weight to power issues these days. But will they inevitably hit a wall in accounting for ultra-low-power considerations? Performance, power, and area are the traditional attributes in chip design. Area was originally the main priority, with feature sizes constantly shrinking according to Moore's Law. Performance was in the saddle for many years. Microprocessors had to be brawnier and faster all the time.

Faster chips, more affordable storage, and open libraries are giving neural network new momentum, and companies are now in the process of figuring out how to optimize it across a variety of markets. The roots of neural networking stretch back to the late 1940s with Claude Shannon's Information Theory, but until several years ago this technology made relatively slow progress. The rush toward autonomous vehicles -- which relies on neural networking to collect data from many sensors -- changed all of that. Work is underway by established companies, startups, and universities around the globe, and funding is pouring into neural networking, as well as related markets such as embedded vision, machine learning, and artificial intelligence. "Mass market economics, increased processing power and improving computational vision techniques equals opportunities for new mass markets to be created," said Tim Ramsdale, general manager of the Imaging and Vision Group at ARM. "But all of this has to be done in real time. Having lights turn on as soon as you appear at the door is critical. That means a minimum of 30 frames per second, and preferably 60 frames per second. To do that you have to have processing at the edge, and processing at the edge means low power."

Following on the heels of their announcement a few weeks ago about their FLIR partnership, Movidius is making another pretty significant announcement regarding their Myriad 2 processor. They've incorporated it into a new USB device called the Fathom Neural Compute Stick. You can plug the Fathom into any USB-capable device (computer, camera, GoPro, Raspberry Pi, Arduino, etc) and that device can become "smarter" in the sense that it can utilize the Myriad 2 processor inside of it to become an input for a neural network (I'll come back to all this). Essentially, it means a device with the Fathom plugged into it can react cognitively or intelligently, based on the things it sees with its camera (via computer vision) or data it processes from another source. A device using it can make its own decisions depending on its programming.

Following on the heels of their announcement a few weeks ago about their FLIR partnership, Movidius is making another pretty significant announcement regarding their Myriad 2 processor. They've incorporated it into a new USB device called the Fathom Neural Compute Stick. You can plug the Fathom into any USB-capable device (computer, camera, GoPro, Raspberry Pi, Arduino, etc) and that device can become "smarter" in the sense that it can utilize the Myriad 2 processor inside of it to become an input for a neural network (I'll come back to all this). Essentially, it means a device with the Fathom plugged into it can react cognitively or intelligently, based on the things it sees with its camera (via computer vision) or data it processes from another source. A device using it can make its own decisions depending on its programming.